Papers with distributed representations
Improving Knowledge Graph Embedding Using Simple Constraints (P18-1)
Copied to clipboard
| Challenge: | Recent efforts focused on designing more complicated models or incorporating extra information beyond triples. |
| Approach: | They propose to use non-negativity constraints on entity representations and approximate entailment constraints on relation representations to improve KG embedding. |
| Outcome: | The proposed model outperforms baseline models on WordNet, Freebase, and DBpedia. |
A New Surprise Measure for Extracting Interesting Relationships between Persons (2021.eacl-demos)
Copied to clipboard
| Challenge: | Interesting facts are useful information for a variety of important tasks. |
| Approach: | They propose a method that extracts all personal relationships from dependency trees and calculates surprise scores for distributed representations of the extracted relationships in an unsupervised manner. |
| Outcome: | The proposed method extracts all personal relationships from dependency trees for the texts and calculates surprise scores for distributed representations of the extracted relationships in an unsupervised manner. |
Self-supervised Post-processing Method to Enrich Pretrained Word Vectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods that use external resources to make word vectors specialize depend on the lexicon. |
| Approach: | They propose a self-supervised extension of extrofitting by its own word vector distribution. |
| Outcome: | The proposed method improves word similarity embeddings on similarity tasks without external resources. |
Leveraging distributed representations and lexico-syntactic fixedness for token-level prediction of the idiomaticity of English verb-noun combinations (P18-2)
Copied to clipboard
| Challenge: | Verb-noun combinations (VNCs) are ambiguous between literal and idiomatic usages in English. |
| Approach: | They propose and evaluate models for classifying verb-noun combinations as idiomatic or literal, based on averaging word embeddings and a variety of approaches to forming distributed representations. |
| Outcome: | The proposed model outperforms a previous model based on skip-thoughts and averaging word embeddings. |
Syntactic Patterns Improve Information Extraction for Medical Search (N18-2)
Copied to clipboard
| Challenge: | Medical professionals search the literature by specifying the type of patients, the medical intervention(s) and the outcome measure(s). |
| Approach: | They propose to exploit the availability of structured abstracts to extract medically relevant information from syntactic patterns. |
| Outcome: | The proposed models differ from the constituent unigrams in the extracted patterns, suggesting that they capture contextual information that is otherwise lost. |
Distant Supervision from Disparate Sources for Low-Resource Part-of-Speech Tagging (D18-1)
Copied to clipboard
| Challenge: | Low-resource languages lack manual annotated data to learn basic models such as part-of-speech (POS) taggers. |
| Approach: | They propose a cross-lingual neural part-of-speech tagger that learns from disparate sources of distant supervision in a uniform framework. |
| Outcome: | The proposed model scales to hundreds of low-resource languages without access to gold annotated data. |
Data Augmentation for Hypernymy Detection (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for supervised inference have limited quality training data. |
| Approach: | They propose two techniques which generate new training examples from existing ones . they combine linguistic principles of hypernym transitivity and intersective modifier-noun composition . |
| Outcome: | The proposed techniques generate new training examples from existing datasets. |
Learning Cross-lingual Distributed Logical Representations for Semantic Parsing (P18-2)
Copied to clipboard
| Challenge: | Recent research efforts have looked into the problem of learning semantic parsers in a multilingual setup, but how to improve the performance of a monolingual semantic parsed system remains a research question that is under-explored. |
| Approach: | They propose to use data annotated in different languages to learn distributed representations of logical forms for improving a monolingual semantic parser. |
| Outcome: | The proposed method improves on the standard multilingual GeoQuery dataset. |
The Feasibility of Embedding Based Automatic Evaluation for Single Document Summarization (D19-1)
Copied to clipboard
| Challenge: | Existing evaluation methods for summarization systems measure semantic overlap between a system summary and a human reference on word-string level. |
| Approach: | They propose to use distributed representations to evaluate system summary and human reference on word-string level. |
| Outcome: | The proposed representations outperform ROUGE on recent corpora but are less good on test data used in previous studies. |
Construction of a Japanese Word Similarity Dataset (L18-1)
Copied to clipboard
| Challenge: | evaluating distributed word representations in languages that do not have such resources is difficult . et al., 2015: distributed word represent a sparse vector indicating the word itself or the context of the word. |
| Approach: | They constructed a Japanese word similarity dataset to evaluate distributed representations in Japanese. |
| Outcome: | a Japanese word similarity dataset is the first resource that can be used to evaluate distributed representations in Japanese . the dataset contains various parts of speech and includes rare words in addition to common words . |
Multilingual Factor Analysis (P19-1)
Copied to clipboard
| Challenge: | Existing methods for multilingual word embeddings are based on the observation that word embeds exhibit similar structures across languages. |
| Approach: | They propose a latent variable-based model that fits a multilingual dictionary to learn multilingual word representations offline. |
| Outcome: | The proposed model is robust to noise in the embedding space making it suitable for distributed representations learned from noisy corpora. |
Distributed Representations of Emotion Categories in Emotion Space (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on emotion detection focus on how to improve performance of models . however, emotion relations are ignored in one-hot representations . |
| Approach: | They propose a framework to learn distributed representations for emotion categories in emotion space from a given emotion classification dataset. |
| Outcome: | The proposed representations can express emotion relations much better than word vectors in semantic space. |
Entity-Duet Neural Ranking: Understanding the Role of Knowledge Graph Semantics in Neural Information Retrieval (P18-1)
Copied to clipboard
| Challenge: | Entity-oriented search and neural-IR push the boundary of search engines from two different aspects. |
| Approach: | They propose an Entity-Duet Neural Ranking Model which integrates knowledge graphs into neural search systems. |
| Outcome: | The proposed model improves generalization ability of neural ranking models on a commercial search log. |
Open Information Extraction from Question-Answer Pairs (N19-1)
Copied to clipboard
| Challenge: | Existing work on OpenIE extracts structured data from sentences . a system for extracting tuples from question-answer pairs solves this problem . |
| Approach: | They propose a system for extracting tuples from question-answer pairs . they use distributed representations of a question and an answer to generate knowledge facts . |
| Outcome: | The proposed system extracts meaningful structured tuples from question-answer pairs . it can find new and interesting facts to extend knowledge bases, the authors show . |
NeuSTIP: A Neuro-Symbolic Model for Link and Time Prediction in Temporal Knowledge Graphs (2023.emnlp-main)
Copied to clipboard
| Challenge: | Temporal Knowledge Graphs (KGs) are factual information repositories where a fact is associated with a time interval. |
| Approach: | They propose a temporal NS model for knowledge graph completion that performs link prediction and time interval prediction in a TKG. |
| Outcome: | The proposed model shows competitive performance on link prediction and time prediction. |
RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to disentangle individual neurons from multiple high-level concepts are not yet benchmarked. |
| Approach: | They propose a method of Multi-task Distributed Alignment Search that allows to find distributed representations satisfying multiple causal criteria. |
| Outcome: | The proposed method achieves state-of-the-art on the target language model with Llama2-7B . |
Meta-CQG: A Meta-Learning Framework for Complex Question Generation over Knowledge Bases (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods train one encoder-decoder-based model to fit all questions . however, such a one-size-fits-all strategy may not perform well for complex questions involving multiple KB relations or functional constraints. |
| Approach: | They propose a meta-learning framework for complex question generation over knowledge bases . they propose he meta-trained generator can acquire universal meta-knowledge . |
| Outcome: | The proposed framework can acquire universal and transferable meta-knowledge and quickly adapt to long-tailed samples under different dimensions. |
Enhancing Unsupervised Semantic Parsing with Distributed Contextual Representations (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to learn models on corpus of pairs of sentences require labor-intensive annotation. |
| Approach: | They propose to leverage distributed contextual word and phrase representations pre-trained on unlabelled texts to deal with homonymy and polysemy. |
| Outcome: | The proposed model achieves better accuracy on question-answering and relation extraction tasks. |